Apriel-H1-15b-Thinker is a hybrid inference model with 15 billion parameters, combining the Transformer attention mechanism and the Mamba state space layer, and performs excellently in inference, mathematics, and coding, with high efficiency and scalability.

Natural Language ProcessingTransformers

i3-80M is an innovative hybrid architecture language model that combines the advantages of convolutional/recurrent layers and full attention layers. The model adopts RWKV-style time mixing and Mamba state space dynamics to achieve efficient sequence processing in the early layers and uses the standard multi-head attention mechanism in the deep layers for complex pattern recognition.

Natural Language ProcessingTransformersEnglish

Falcon-H1 is a new family of large language models developed by the Technology Innovation Institute of the United Arab Emirates. It adopts a hybrid architecture that combines the Transformer attention mechanism and the state space model (SSMs), with excellent long-context memory ability and computational efficiency. This series of models offers various configurations with parameters ranging from 0.5B to 34B, and performs excellently in tasks such as inference, mathematics, and multilingual tasks.

Natural Language ProcessingTransformersMultiple Languages

Zamba2-2.7B is a hybrid model composed of state space and Transformer modules, using the Mamba2 module and shared attention module, featuring high performance and low latency.

Natural Language ProcessingTransformers

Zamba-7B-v1-phase1 is a hybrid architecture combining state space model Mamba with Transformer, using Mamba as the backbone network and sharing one Transformer layer every six modules, trained via next-token prediction.

Natural Language ProcessingTransformers